Getting started with your Lab Sandbox
You can find important course-specific tips and notes for your Lab Sandbox in this
quick guide to use throughout your course. You’ll be able to reference this at any time
or visit the Learner Help Center
for more info.
Throughout this course, you'll encounter datasets which are hosted on other websites or are linked from the course instructional materials. If you'd like to complete your work in the lab sandbox environment, please download these datasets from their listed websites and upload the data files directly into your RStudio lab environment. Lab Sandboxes have limited access to external sites, so uploading your data files directly will help ensure you do not encounter any access errors.
What tools are already installed in my sandbox environment?
- Python==3.7.6
- turicreate==6.4.1
- scikit-learn(same as sklearn)==0.22.2.post1
- pandas==1.0.3
- numpy==1.18.4
- matplotlib==3.2.1
- unzip for uploading/unpacking data and files via Jupyter's built-in Terminal
Additional Course Specific Notes:
- IMPORTANT: To optimize your notebook performance, please shut down each of your
open/running Jupyter Notebooks after you complete them within your Lab.
This will provide full and dedicated system resources to the current lab assignment
you are working on in your Lab Sandbox. If you experience any unexpected slow
processing in your notebook, try to restart your Lab kernel and only open that single
notebook to complete your work. You can learn more about restarting your Lab Sandbox
and troubleshooting your lab in our Learner Help Center article here. - Ensure you are using Python3 to complete the assessments. In Python3, the print statement syntax
is print("Hello") rather than print "Hello" as stated in the solution files - Upload the unzipped files to the Labs Sandbox.
- It is highly advised to practice with turicreate if you use this sandbox as it
can be used on large datasets.
Turicreate and python3 will be the preferred/supported route for the sandbox,
to enable a newer version of python and avoid compatibility issues between versions.
If you prefer to have Python2 installed in your local system, you can use Sframe or graphlab locally as well.
You are still able to complete all assignments locally if you would prefer using other tools. - As an important note, the sandbox environment
has limited access to the internet, so you will not be
able to install additional packages into this sandbox. However, the required dependencies for the Python3,
Jupyter Notebook, and turicreate path are pre-installed. This should still allow you to complete your coursework
in the sandbox if you are unable to complete it locally. If you find that you are blocked using available tools in this
sandbox at any point, please reach out to Coursera through the Learner Help Center with your feedback! - Decide on which tool you'd like to use at the very beginning of the course. It is highly advisable
not to switch between turicreate and other Python libraries such as sklearn or Pandas. - If anything is unclear regarding the course content, please go through the
Discussion board and see if your question has been answered. If you have any issues
with the Jupyter enviornment itself, please contact Coursera through the Learner Help Center.
Important Note:
Happy Learning!
In [1]:
! pip install turicreate==6.4.1
In [ ]:
!unzip people_wiki.sframe.zip
In [3]:
from __future__ import print_function # to conform python 2.x print to python 3.x
import turicreate
import matplotlib.pyplot as plt
import numpy as np
%matplotlib inline
In [7]:
wiki = turicreate.SFrame('people_wiki.sframe')
In [8]:
wiki
Out[8]:
In [89]:
wiki['word_count'] = turicreate.text_analytics.count_words(wiki['text'])
In [10]:
model = turicreate.nearest_neighbors.create(wiki, label='name', features=['word_count'],
method='brute_force', distance='euclidean')
In [11]:
model.query(wiki[wiki['name']=='Barack Obama'], label='name', k=10)
Out[11]:
In [12]:
def top_words(name):
"""
Get a table of the most frequent words in the given person's wikipedia page.
"""
row = wiki[wiki['name'] == name]
word_count_table = row[['word_count']].stack('word_count', new_column_name=['word','count'])
return word_count_table.sort('count', ascending=False)
In [13]:
obama_words = top_words('Barack Obama')
obama_words
Out[13]:
In [14]:
barrio_words = top_words('Francisco Barrio')
barrio_words
Out[14]:
In [15]:
combined_words = obama_words.join(barrio_words, on='word')
combined_words
Out[15]:
In [16]:
combined_words = combined_words.rename({'count':'Obama', 'count.1':'Barrio'})
combined_words
Out[16]:
In [19]:
combined_words.sort('Obama', ascending=False)[:5]
Out[19]:
In [24]:
common_words = set(combined_words.sort('Obama', ascending=False)[:5]['word']) # YOUR CODE HERE
def has_top_words(word_count_vector):
# extract the keys of word_count_vector and convert it to a set
unique_words = set(word_count_vector.keys()) # YOUR CODE HERE
# return True if common_words is a subset of unique_words
# return False otherwise
return common_words.issubset(unique_words) # YOUR CODE HERE
wiki['has_top_words'] = wiki['word_count'].apply(has_top_words)
# use has_top_words column to answer the quiz question
... # YOUR CODE HERE
wiki['has_top_words'].sum()
Out[24]:
In [25]:
len(wiki['has_top_words'])
Out[25]:
In [26]:
print('Output from your function:', has_top_words(wiki[32]['word_count']))
print('Correct output: True')
print('Also check the length of unique_words. It should be 167')
print(len(wiki[32]['word_count']))
In [27]:
print('Output from your function:', has_top_words(wiki[33]['word_count']))
print('Correct output: False')
print('Also check the length of unique_words. It should be 188')
print(len(wiki[33]['word_count']))
In [31]:
o = wiki[wiki['name'] == 'Barack Obama']['word_count'][0]
b = wiki[wiki['name'] == 'George W. Bush']['word_count'][0]
j = wiki[wiki['name'] == 'Joe Biden']['word_count'][0]
turicreate.toolkits.distances.euclidean(o, b)
Out[31]:
In [32]:
turicreate.toolkits.distances.euclidean(o, j)
Out[32]:
In [33]:
turicreate.toolkits.distances.euclidean(j, b)
Out[33]:
In [34]:
bush_words = top_words('George W. Bush')
combined_words = obama_words.join(bush_words, on='word')
combined_words = combined_words.rename({'count':'Obama', 'count.1':'Bush'})
combined_words.sort('Obama', ascending=False)
Out[34]:
In [90]:
wiki['tf_idf'] = turicreate.text_analytics.tf_idf(wiki['word_count'])
In [91]:
model_tf_idf = turicreate.nearest_neighbors.create(wiki, label='name', features=['tf_idf'],
method='brute_force', distance='euclidean')
In [92]:
model_tf_idf.query(wiki[wiki['name'] == 'Barack Obama'], label='name', k=10)
Out[92]:
In [93]:
def top_words_tf_idf(name):
row = wiki[wiki['name'] == name]
word_count_table = row[['tf_idf']].stack('tf_idf', new_column_name=['word','weight'])
return word_count_table.sort('weight', ascending=False)
In [94]:
obama_tf_idf = top_words_tf_idf('Barack Obama')
obama_tf_idf
Out[94]:
In [95]:
schiliro_tf_idf = top_words_tf_idf('Phil Schiliro')
schiliro_tf_idf
Out[95]:
In [97]:
combined_words = obama_tf_idf.join(schiliro_tf_idf, on='word')
combined_words = combined_words.rename({'weight':'Obama', 'weight.1':'Schiliro'})
combined_words.sort('Obama', ascending=False)
Out[97]:
In [98]:
common_words = set(combined_words.sort('Obama', ascending=False)[:5]['word']) # YOUR CODE HERE
def has_top_words(word_count_vector):
# extract the keys of word_count_vector and convert it to a set
unique_words = set(word_count_vector.keys()) # YOUR CODE HERE
# return True if common_words is a subset of unique_words
# return False otherwise
return common_words.issubset(unique_words) # YOUR CODE HERE
wiki['has_top_words'] = wiki['word_count'].apply(has_top_words)
# use has_top_words column to answer the quiz question
... # YOUR CODE HERE
wiki['has_top_words'].sum()
# use has_top_words column to answer the quiz question
# ... # YOUR CODE HERE
Out[98]:
In [99]:
common_words
Out[99]:
In [100]:
biden_tf_idf = top_words_tf_idf('Joe Biden')
In [101]:
biden_tf_idf
Out[101]:
In [102]:
combined_words = obama_tf_idf.join(biden_tf_idf, on='word')
combined_words['weight'], combined_words['weight.1']
Out[102]:
In [104]:
combined_words
Out[104]:
In [111]:
turicreate.toolkits.distances.euclidean(list(combined_words['weight']), list(combined_words['weight.1']))
Out[111]:
In [109]:
turicreate.toolkits.distances.euclidean(wiki[wiki['name'] == 'Barack Obama']['tf_idf'][0],
wiki[wiki['name'] == 'Joe Biden']['tf_idf'][0])
Out[109]:
In [44]:
model_tf_idf.query(wiki[wiki['name'] == 'Barack Obama'], label='name', k=10)
Out[44]:
In [45]:
def compute_length(row):
return len(row['text'].split(' '))
wiki['length'] = wiki.apply(compute_length)
In [46]:
nearest_neighbors_euclidean = model_tf_idf.query(wiki[wiki['name'] == 'Barack Obama'], label='name', k=100)
nearest_neighbors_euclidean = nearest_neighbors_euclidean.join(wiki[['name', 'length']], on={'reference_label':'name'})
In [47]:
nearest_neighbors_euclidean.sort('rank')
Out[47]:
In [48]:
plt.figure(figsize=(10.5,4.5))
plt.hist(wiki['length'], 50, color='k', edgecolor='None', histtype='stepfilled', normed=True,
label='Entire Wikipedia', zorder=3, alpha=0.8)
plt.hist(nearest_neighbors_euclidean['length'], 50, color='r', edgecolor='None', histtype='stepfilled', normed=True,
label='100 NNs of Obama (Euclidean)', zorder=10, alpha=0.8)
plt.axvline(x=wiki['length'][wiki['name'] == 'Barack Obama'][0], color='k', linestyle='--', linewidth=4,
label='Length of Barack Obama', zorder=2)
plt.axvline(x=wiki['length'][wiki['name'] == 'Joe Biden'][0], color='g', linestyle='--', linewidth=4,
label='Length of Joe Biden', zorder=1)
plt.axis([0, 1000, 0, 0.04])
plt.legend(loc='best', prop={'size':15})
plt.title('Distribution of document length')
plt.xlabel('# of words')
plt.ylabel('Percentage')
plt.rcParams.update({'font.size':16})
plt.tight_layout()
In [ ]:
model2_tf_idf = turicreate.nearest_neighbors.create(wiki, label='name', features=['tf_idf'],
method='brute_force', distance='cosine')
In [ ]:
nearest_neighbors_cosine = model2_tf_idf.query(wiki[wiki['name'] == 'Barack Obama'], label='name', k=100)
nearest_neighbors_cosine = nearest_neighbors_cosine.join(wiki[['name', 'length']], on={'reference_label':'name'})
In [ ]:
nearest_neighbors_cosine.sort('rank')
In [ ]:
plt.figure(figsize=(10.5,4.5))
plt.figure(figsize=(10.5,4.5))
plt.hist(wiki['length'], 50, color='k', edgecolor='None', histtype='stepfilled', normed=True,
label='Entire Wikipedia', zorder=3, alpha=0.8)
plt.hist(nearest_neighbors_euclidean['length'], 50, color='r', edgecolor='None', histtype='stepfilled', normed=True,
label='100 NNs of Obama (Euclidean)', zorder=10, alpha=0.8)
plt.hist(nearest_neighbors_cosine['length'], 50, color='b', edgecolor='None', histtype='stepfilled', normed=True,
label='100 NNs of Obama (cosine)', zorder=11, alpha=0.8)
plt.axvline(x=wiki['length'][wiki['name'] == 'Barack Obama'][0], color='k', linestyle='--', linewidth=4,
label='Length of Barack Obama', zorder=2)
plt.axvline(x=wiki['length'][wiki['name'] == 'Joe Biden'][0], color='g', linestyle='--', linewidth=4,
label='Length of Joe Biden', zorder=1)
plt.axis([0, 1000, 0, 0.04])
plt.legend(loc='best', prop={'size':15})
plt.title('Distribution of document length')
plt.xlabel('# of words')
plt.ylabel('Percentage')
plt.rcParams.update({'font.size': 16})
plt.tight_layout()
In [49]:
sf = turicreate.SFrame({'text': ['democratic governments control law in response to popular act']})
sf['word_count'] = turicreate.text_analytics.count_words(sf['text'])
encoder = turicreate.toolkits._feature_engineering.TFIDF(features=['word_count'], output_column_prefix='tf_idf')
encoder.fit(wiki)
sf = encoder.transform(sf)
sf
Out[49]:
In [50]:
tweet_tf_idf = sf[0]['tf_idf.word_count']
tweet_tf_idf
Out[50]:
In [51]:
obama = wiki[wiki['name'] == 'Barack Obama']
obama
Out[51]:
In [52]:
obama_tf_idf = obama[0]['tf_idf']
turicreate.toolkits.distances.cosine(obama_tf_idf, tweet_tf_idf)
Out[52]:
In [ ]:
model2_tf_idf.query(obama, label='name', k=10)
In [ ]:
In [3]:
from __future__ import print_function # to conform python 2.x print to python 3.x
import numpy as np
import turicreate
from scipy.sparse import csr_matrix
from sklearn.metrics.pairwise import pairwise_distances
import time
from copy import copy
import matplotlib.pyplot as plt
%matplotlib inline
'''compute norm of a sparse vector
Thanks to: Jaiyam Sharma'''
def norm(x):
sum_sq=x.dot(x.T)
norm=np.sqrt(sum_sq)
return(norm)
In [4]:
wiki = turicreate.SFrame('people_wiki.sframe/')
In [5]:
wiki = wiki.add_row_number()
In [6]:
wiki['tf_idf'] = turicreate.text_analytics.tf_idf(wiki['text'])
wiki.head()
Out[6]:
In [7]:
def sframe_to_scipy(x, column_name):
'''
Convert a dictionary column of an SFrame into a sparse matrix format where
each (row_id, column_id, value) triple corresponds to the value of
x[row_id][column_id], where column_id is a key in the dictionary.
Example
>>> sparse_matrix, map_key_to_index = sframe_to_scipy(sframe, column_name)
'''
assert type(x[column_name][0]) == dict, \
'The chosen column must be dict type, representing sparse data.'
# Stack will transform x to have a row for each unique (row, key) pair.
x = x.stack(column_name, ['feature', 'value'])
# Map feature words to integers
unique_words = sorted(x['feature'].unique())
mapping = {word:i for i, word in enumerate(unique_words)}
x['feature_id'] = x['feature'].apply(lambda x: mapping[x])
# Create numpy arrays that contain the data for the sparse matrix.
row_id = np.array(x['id'])
col_id = np.array(x['feature_id'])
data = np.array(x['value'])
width = x['id'].max() + 1
height = x['feature_id'].max() + 1
# Create a sparse matrix.
mat = csr_matrix((data, (row_id, col_id)), shape=(width, height))
return mat, mapping
In [8]:
%%time
corpus, mapping = sframe_to_scipy(wiki, 'tf_idf')
In [9]:
assert corpus.shape == (59071, 547979)
print('Check passed correctly!')
In [10]:
def generate_random_vectors(dim, n_vectors):
return np.random.randn(dim, n_vectors)
In [11]:
# Generate 16 random vectors of dimension 547979
np.random.seed(0)
n_vectors = 16
random_vectors = generate_random_vectors(corpus.shape[1], n_vectors)
random_vectors.shape
Out[11]:
In [12]:
sample = corpus[0] # vector of tf-idf values for document 0
bin_indices_bits = sample.dot(random_vectors[:,0]) >= 0
bin_indices_bits
Out[12]:
In [13]:
sample.dot(random_vectors[:, 1]) >= 0 # True if positive sign; False if negative sign
Out[13]:
In [14]:
sample.dot(random_vectors) >= 0 # should return an array of 16 True/False bits
Out[14]:
In [15]:
np.array(sample.dot(random_vectors) >= 0, dtype=int) # display index bits in 0/1's
Out[15]:
In [16]:
corpus[0:2].dot(random_vectors) >= 0 # compute bit indices of first two documents
Out[16]:
In [17]:
corpus.dot(random_vectors) >= 0 # compute bit indices of ALL documents
Out[17]:
In [18]:
index_bits = (sample.dot(random_vectors) >= 0)
powers_of_two = (1 << np.arange(15, -1, -1))
print(index_bits)
print(powers_of_two)
print(index_bits.dot(powers_of_two))
In [19]:
index_bits = sample.dot(random_vectors) >= 0
index_bits.dot(powers_of_two)
Out[19]:
In [20]:
from collections import defaultdict
def train_lsh(data, n_vectors, seed=None):
if seed is not None:
np.random.seed(seed)
dim = data.shape[1]
random_vectors = generate_random_vectors(dim, n_vectors)
# Partition data points into bins,
# and encode bin index bits into integers
bin_indices_bits = data.dot(random_vectors) >= 0
powers_of_two = 1 << np.arange(n_vectors - 1, -1, step=-1)
bin_indices = bin_indices_bits.dot(powers_of_two)
# Update `table` so that `table[i]` is the list of document ids with bin index equal to i
table = defaultdict(list)
for idx, bin_index in enumerate(bin_indices):
# Fetch the list of document ids associated with the bin and add the document id to the end.
# data_index: document ids
# append() will add a list of document ids to table dict() with key as bin_index
table[bin_index].append(idx) # YOUR CODE HERE
# Note that we're storing the bin_indices here
# so we can do some ad-hoc checking with it,
# this isn't actually required
model = {'data': data,
'table': table,
'random_vectors': random_vectors,
'bin_indices': bin_indices,
'bin_indices_bits': bin_indices_bits}
return model
In [21]:
def compare_bits(model, id_1, id_2):
bits1 = model['bin_indices_bits'][id_1]
bits2 = model['bin_indices_bits'][id_2]
print('Number of agreed bits: ', np.sum(bits1 == bits2))
return np.sum(bits1 == bits2)
In [22]:
model = train_lsh(corpus, 16, seed=475)
obama_id = wiki[wiki['name'] == 'Barack Obama']['id'][0]
biden_id = wiki[wiki['name'] == 'Joe Biden']['id'][0]
similariy = compare_bits(model, obama_id, biden_id)
In [23]:
# This function will help us get similar items, given the id
def get_similarity_items(X_tfidf, item_id, topn=5):
"""
Get the top similar items for a given item id.
The similarity measure here is based on cosine distance.
"""
query = X_tfidf[item_id]
scores = X_tfidf.dot(query.T).toarray().ravel()
best = np.argpartition(scores, -topn)[-topn:]
similar_items = sorted(zip(best, scores[best]), key=lambda x: -x[1])
similar_item_ids = [similar_item for similar_item, _ in similar_items]
print("Similar items to id: {}".format(item_id))
for _id in similar_item_ids:
print(wiki[_id]['name'])
print('\n')
return similar_item_ids
In [24]:
wiki[wiki['name'] == 'Barack Obama']
Out[24]:
In [25]:
obama_id
Out[25]:
In [26]:
s = ''.join(map(str,model['bin_indices_bits'][obama_id].astype(int)))
sum(int(c) * (2 ** i) for i, c in enumerate(s[::-1]))
Out[26]:
In [27]:
#s = '1110'
#sum(int(c) * (2 ** i) for i, c in enumerate(s[::-1]))
In [28]:
wiki[wiki['name'] == 'Joe Biden']
Out[28]:
In [29]:
so = ''.join(map(str,model['bin_indices_bits'][obama_id].astype(int)))
sb = ''.join(map(str,model['bin_indices_bits'][biden_id].astype(int)))
sum([so[i]==sb[i] for i in range(len(so))])
Out[29]:
In [30]:
jones_id = wiki[wiki['name']=='Wynn Normington Hugh-Jones']['id'][0]
compare_bits(model, obama_id, jones_id)
Out[30]:
In [31]:
model['bin_indices'][obama_id]
Out[31]:
In [32]:
model['table'][model['bin_indices'][obama_id]]
Out[32]:
In [33]:
doc_ids = list(model['table'][model['bin_indices'][35817]])
doc_ids.remove(35817) # display documents other than Obama
docs = wiki.filter_by(values=doc_ids, column_name='id') # filter by id column
docs
Out[33]:
In [34]:
res = compare_bits(model, obama_id, docs[0]['id']), compare_bits(model, obama_id, biden_id)
In [35]:
from itertools import combinations
In [36]:
num_vector = 16
search_radius = 3
for diff in combinations(range(num_vector), search_radius):
print(diff)
In [37]:
def search_nearby_bins(query_bin_bits, table, search_radius=2, initial_candidates=set()):
"""
For a given query vector and trained LSH model, return all candidate neighbors for
the query among all bins within the given search radius.
Example usage
-------------
>>> model = train_lsh(corpus, num_vector=16, seed=143)
>>> q = model['bin_index_bits'][0] # vector for the first document
>>> candidates = search_nearby_bins(q, model['table'])
"""
num_vector = len(query_bin_bits)
powers_of_two = 1 << np.arange(num_vector-1, -1, -1)
# Allow the user to provide an initial set of candidates.
candidate_set = copy(initial_candidates)
for different_bits in combinations(range(num_vector), search_radius):
# Flip the bits (n_1,n_2,...,n_r) of the query bin to produce a new bit vector.
## Hint: you can iterate over a tuple like a list
alternate_bits = copy(query_bin_bits)
for i in different_bits:
alternate_bits[i] = 1 - alternate_bits[i] # YOUR CODE HERE
# Convert the new bit vector to an integer index
nearby_bin = alternate_bits.dot(powers_of_two)
# Fetch the list of documents belonging to the bin indexed by the new bit vector.
# Then add those documents to candidate_set
# Make sure that the bin exists in the table!
# Hint: update() method for sets lets you add an entire list to the set
if nearby_bin in table:
more_docs = table[nearby_bin] # Get all document_ids of the bin
... # YOUR CODE HERE: Update candidate_set with the documents in this bin.
candidate_set.update(more_docs)
return candidate_set
In [38]:
obama_bin_index = model['bin_indices_bits'][35817] # bin index of Barack Obama
candidate_set = search_nearby_bins(obama_bin_index, model['table'], search_radius=0)
if candidate_set == set({35817, 54743}):
print('Passed test')
else:
print('Check your code')
print('List of documents in the same bin as Obama: {}'.format(candidate_set))
In [39]:
candidate_set = search_nearby_bins(obama_bin_index, model['table'], search_radius=1, initial_candidates=candidate_set)
if candidate_set == set({42243, 28804, 1810, 48919, 24478, 31010, 7331, 23716, 51108, 48040, 36266, 33200, 25023, 23617, 54743, 34910, 35817, 34159, 14451, 23926, 39032, 12028, 43775}):
print('Passed test')
else:
print('Check your code')
print(candidate_set)
In [40]:
def query(vec, model, k, max_search_radius):
data = model['data']
table = model['table']
random_vectors = model['random_vectors']
num_vector = random_vectors.shape[1]
# Compute bin index for the query vector, in bit representation.
bin_index_bits = (vec.dot(random_vectors) >= 0).flatten()
# Search nearby bins and collect candidates
candidate_set = set()
for search_radius in range(max_search_radius+1):
candidate_set = search_nearby_bins(bin_index_bits, table, search_radius, initial_candidates=candidate_set)
# Sort candidates by their true distances from the query
nearest_neighbors = turicreate.SFrame({'id':candidate_set})
candidates = data[np.array(list(candidate_set)),:]
nearest_neighbors['distance'] = pairwise_distances(candidates, vec, metric='cosine').flatten()
return nearest_neighbors.topk('distance', k, reverse=True), len(candidate_set)
In [41]:
query(corpus[35817,:], model, k=10, max_search_radius=3)
Out[41]:
In [42]:
query(corpus[35817,:], model, k=10, max_search_radius=3)[0].join(wiki[['id', 'name']], on='id').sort('distance')
Out[42]:
In [43]:
wiki[wiki['name']=='Barack Obama']
Out[43]:
In [45]:
%%time
num_candidates_history = []
query_time_history = []
max_distance_from_query_history = []
min_distance_from_query_history = []
average_distance_from_query_history = []
for max_search_radius in range(17):
start=time.time()
result, num_candidates = query(corpus[35817,:], model, k=10,
max_search_radius=max_search_radius)
end=time.time()
query_time = end-start
print('Radius:', max_search_radius)
print(result.join(wiki[['id', 'name']], on='id').sort('distance'))
#res = list(result.join(wiki[['id', 'name']], on='id')['distance'])
#print(res)
#print(np.mean(res))
average_distance_from_query = result['distance'][1:].mean()
max_distance_from_query = result['distance'][1:].max()
min_distance_from_query = result['distance'][1:].min()
print(average_distance_from_query)
num_candidates_history.append(num_candidates)
query_time_history.append(query_time)
average_distance_from_query_history.append(average_distance_from_query)
max_distance_from_query_history.append(max_distance_from_query)
min_distance_from_query_history.append(min_distance_from_query)
In [99]:
plt.figure(figsize=(7,4.5))
plt.plot(num_candidates_history, linewidth=4)
plt.xlabel('Search radius')
plt.ylabel('# of documents searched')
plt.rcParams.update({'font.size':16})
plt.tight_layout()
plt.figure(figsize=(7,4.5))
plt.plot(query_time_history, linewidth=4)
plt.xlabel('Search radius')
plt.ylabel('Query time (seconds)')
plt.rcParams.update({'font.size':16})
plt.tight_layout()
plt.figure(figsize=(7,4.5))
plt.plot(average_distance_from_query_history, linewidth=4, label='Average of 10 neighbors')
plt.plot(max_distance_from_query_history, linewidth=4, label='Farthest of 10 neighbors')
plt.plot(min_distance_from_query_history, linewidth=4, label='Closest of 10 neighbors')
plt.xlabel('Search radius')
plt.ylabel('Cosine distance of neighbors')
plt.legend(loc='best', prop={'size':15})
plt.rcParams.update({'font.size':16})
plt.tight_layout()
In [ ]:
In [2]:
from __future__ import print_function # to conform python 2.x print to python 3.x
import turicreate
import matplotlib.pyplot as plt
import numpy as np
import sys
import os
from scipy.sparse import csr_matrix
from sklearn.preprocessing import OneHotEncoder, LabelEncoder
%matplotlib inline
In [3]:
wiki = turicreate.SFrame('people_wiki.sframe/')
In [4]:
wiki['tf_idf'] = turicreate.text_analytics.tf_idf(wiki['text'])
In [5]:
def sframe_to_scipy(x, column_name):
'''
Convert a dictionary column of an SFrame into a sparse matrix format where
each (row_id, column_id, value) triple corresponds to the value of
x[row_id][column_id], where column_id is a key in the dictionary.
Example
>>> sparse_matrix, map_key_to_index = sframe_to_scipy(sframe, column_name)
'''
assert type(x[column_name][0]) == dict, \
'The chosen column must be dict type, representing sparse data.'
# 1. Add a row number (id)
x = x.add_row_number()
# 2. Stack will transform x to have a row for each unique (row, key) pair.
x = x.stack(column_name, ['feature', 'value'])
# Map feature words to integers
unique_words = sorted(x['feature'].unique())
mapping = {word:i for i, word in enumerate(unique_words)}
x['feature_id'] = x['feature'].apply(lambda x: mapping[x])
# Create numpy arrays that contain the data for the sparse matrix.
row_id = np.array(x['id'])
col_id = np.array(x['feature_id'])
data = np.array(x['value'])
width = x['id'].max() + 1
height = x['feature_id'].max() + 1
# Create a sparse matrix.
mat = csr_matrix((data, (row_id, col_id)), shape=(width, height))
return mat, mapping
In [6]:
%%time
# The conversion will take about a minute or two.
tf_idf, map_index_to_word = sframe_to_scipy(wiki, 'tf_idf')
In [7]:
tf_idf.shape
Out[7]:
In [8]:
from sklearn.preprocessing import normalize
tf_idf = normalize(tf_idf)
In [9]:
def get_initial_centroids(data, k, seed=None):
'''Randomly choose k data points as initial centroids'''
if seed is not None: # useful for obtaining consistent results
np.random.seed(seed)
n = data.shape[0] # number of data points
# Pick K indices from range [0, N).
rand_indices = np.random.randint(0, n, k)
# Keep centroids as dense format, as many entries will be nonzero due to averaging.
# As long as at least one document in a cluster contains a word,
# it will carry a nonzero weight in the TF-IDF vector of the centroid.
centroids = data[rand_indices,:].toarray()
return centroids
In [10]:
from sklearn.metrics import pairwise_distances
# Get the TF-IDF vectors for documents 100 through 102.
queries = tf_idf[100:102,:]
# Compute pairwise distances from every data point to each query vector.
dist = pairwise_distances(tf_idf, queries, metric='euclidean')
print(dist)
In [11]:
k = 3
centroids = tf_idf[:k,:]
distances = pairwise_distances(tf_idf, centroids, metric='euclidean')
print(distances)
dist = pairwise_distances(tf_idf[430,:], centroids[1], metric='euclidean')
print(dist)
In [12]:
'''Test cell'''
if np.allclose(dist, pairwise_distances(tf_idf[430,:], tf_idf[1,:])):
print('Pass')
else:
print('Check your code again')
In [13]:
closest_cluster = np.argmin(distances, 1)
closest_cluster
Out[13]:
In [14]:
'''Test cell'''
reference = [list(row).index(min(row)) for row in distances]
if np.allclose(closest_cluster, reference):
print('Pass')
else:
print('Check your code again')
In [15]:
def get_cluster_assignments(data, centroids):
distances = pairwise_distances(data, centroids, metric='euclidean')
return np.argmin(distances, 1)
cluster_assignment = get_cluster_assignments(tf_idf, centroids)
In [16]:
if len(cluster_assignment)==59071 and \
np.array_equal(np.bincount(cluster_assignment), np.array([23061, 10086, 25924])):
print('Pass') # count number of data points for each cluster
else:
print('Check your code again.')
In [17]:
def assign_clusters(data, centroids):
# Compute distances between each data point and the set of centroids:
# Fill in the blank (RHS only)
#distances_from_centroids = ... # YOUR CODE HERE
# Compute cluster assignments for each data point:
# Fill in the blank (RHS only)
cluster_assignment = get_cluster_assignments(data, centroids) # YOUR CODE HERE
return cluster_assignment
In [18]:
if np.allclose(assign_clusters(tf_idf[0:100:10], tf_idf[0:8:2]), np.array([0, 1, 1, 0, 0, 2, 0, 2, 2, 1])):
print('Pass')
else:
print('Check your code again.')
In [19]:
data = np.array([[1., 2., 0.],
[0., 0., 0.],
[2., 2., 0.]])
centroids = np.array([[0.5, 0.5, 0.],
[0., -0.5, 0.]])
In [20]:
cluster_assignment = assign_clusters(data, centroids)
print(cluster_assignment)
In [21]:
cluster_assignment==1
Out[21]:
In [22]:
cluster_assignment==0
Out[22]:
In [23]:
data[cluster_assignment==1]
Out[23]:
In [24]:
data[cluster_assignment==0]
Out[24]:
In [25]:
data[cluster_assignment==0].mean(axis=0)
Out[25]:
In [26]:
def revise_centroids(data, k, cluster_assignment):
new_centroids = []
for i in range(k):
# Select all data points that belong to cluster i. Fill in the blank (RHS only)
member_data_points = data[cluster_assignment == i] # YOUR CODE HERE
# Compute the mean of the data points. Fill in the blank (RHS only)
centroid = np.mean(member_data_points, axis=0) # YOUR CODE HERE
# Convert numpy.matrix type to numpy.ndarray type
centroid = centroid.A1
new_centroids.append(centroid)
new_centroids = np.array(new_centroids)
return new_centroids
In [27]:
result = revise_centroids(tf_idf[0:100:10], 3, np.array([0, 1, 1, 0, 0, 2, 0, 2, 2, 1]))
if np.allclose(result[0], np.mean(tf_idf[[0,30,40,60]].toarray(), axis=0)) and \
np.allclose(result[1], np.mean(tf_idf[[10,20,90]].toarray(), axis=0)) and \
np.allclose(result[2], np.mean(tf_idf[[50,70,80]].toarray(), axis=0)):
print('Pass')
else:
print('Check your code')
In [28]:
def compute_heterogeneity(data, k, centroids, cluster_assignment):
heterogeneity = 0.0
for i in range(k):
# Select all data points that belong to cluster i. Fill in the blank (RHS only)
member_data_points = data[cluster_assignment==i, :]
if member_data_points.shape[0] > 0: # check if i-th cluster is non-empty
# Compute distances from centroid to data points (RHS only)
distances = pairwise_distances(member_data_points, [centroids[i]], metric='euclidean')
squared_distances = distances**2
heterogeneity += np.sum(squared_distances)
return heterogeneity
In [29]:
compute_heterogeneity(data, 2, centroids, cluster_assignment)
Out[29]:
In [30]:
# Fill in the blanks
def kmeans(data, k, initial_centroids, maxiter, record_heterogeneity=None, verbose=False):
'''This function runs k-means on given data and initial set of centroids.
maxiter: maximum number of iterations to run.
record_heterogeneity: (optional) a list, to store the history of heterogeneity as function of iterations
if None, do not store the history.
verbose: if True, print how many data points changed their cluster labels in each iteration'''
centroids = initial_centroids[:]
prev_cluster_assignment = None
for itr in range(maxiter):
if verbose:
print(itr)
# 1. Make cluster assignments using nearest centroids
# YOUR CODE HERE
cluster_assignment = assign_clusters(data, centroids)
# 2. Compute a new centroid for each of the k clusters, averaging all data points assigned to that cluster.
# YOUR CODE HERE
centroids = revise_centroids(data, k, cluster_assignment)
# Check for convergence: if none of the assignments changed, stop
if prev_cluster_assignment is not None and \
(prev_cluster_assignment==cluster_assignment).all():
break
# Print number of new assignments
if prev_cluster_assignment is not None:
num_changed = np.sum(prev_cluster_assignment!=cluster_assignment)
if verbose:
print(' {0:5d} elements changed their cluster assignment.'.format(num_changed))
# Record heterogeneity convergence metric
if record_heterogeneity is not None:
# YOUR CODE HERE
score = compute_heterogeneity(data, k, centroids, cluster_assignment)
record_heterogeneity.append(score)
prev_cluster_assignment = cluster_assignment[:]
return centroids, cluster_assignment
In [31]:
def plot_heterogeneity(heterogeneity, k):
plt.figure(figsize=(7,4))
plt.plot(heterogeneity, linewidth=4)
plt.xlabel('# Iterations')
plt.ylabel('Heterogeneity')
plt.title('Heterogeneity of clustering over time, K={0:d}'.format(k))
plt.rcParams.update({'font.size': 16})
plt.tight_layout()
In [41]:
k = 3
heterogeneity = []
initial_centroids = get_initial_centroids(tf_idf, k, seed=0)
centroids, cluster_assignment = kmeans(tf_idf, k, initial_centroids, maxiter=400,
record_heterogeneity=heterogeneity, verbose=True)
plot_heterogeneity(heterogeneity, k)
In [42]:
np.bincount(cluster_assignment)
Out[42]:
In [34]:
k = 10
heterogeneity = {}
cluster_assignment_dict = {}
import time
start = time.time()
for seed in [0, 20000, 40000, 60000, 80000, 100000, 120000]:
initial_centroids = get_initial_centroids(tf_idf, k, seed)
centroids, cluster_assignment = kmeans(tf_idf, k, initial_centroids, maxiter=400,
record_heterogeneity=None, verbose=False)
# To save time, compute heterogeneity only once in the end
heterogeneity[seed] = compute_heterogeneity(tf_idf, k, centroids, cluster_assignment)
# This is the line we added for the next quiz question
cluster_assignment_dict[seed] = np.bincount(cluster_assignment)
# print('seed={0:06d}, heterogeneity={1:.5f}'.format(seed, heterogeneity[seed]))
# And this is the modified print statement
print('seed={0:06d}, heterogeneity={1:.5f}, cluster_distribution={2}'.format(seed, heterogeneity[seed],
cluster_assignment_dict[seed]))
sys.stdout.flush()
end = time.time()
print(end-start)
In [35]:
def smart_initialize(data, k, seed=None):
'''Use k-means++ to initialize a good set of centroids'''
if seed is not None: # useful for obtaining consistent results
np.random.seed(seed)
centroids = np.zeros((k, data.shape[1]))
# Randomly choose the first centroid.
# Since we have no prior knowledge, choose uniformly at random
idx = np.random.randint(data.shape[0])
centroids[0] = data[idx,:].toarray()
# Compute distances from the first centroid chosen to all the other data points
squared_distances = pairwise_distances(data, centroids[0:1], metric='euclidean').flatten()**2
for i in range(1, k):
# Choose the next centroid randomly, so that the probability for each data point to be chosen
# is directly proportional to its squared distance from the nearest centroid.
# Roughtly speaking, a new centroid should be as far as from ohter centroids as possible.
idx = np.random.choice(data.shape[0], 1, p=squared_distances/sum(squared_distances))
centroids[i] = data[idx,:].toarray()
# Now compute distances from the centroids to all data points
squared_distances = np.min(pairwise_distances(data, centroids[0:i+1], metric='euclidean')**2,axis=1)
return centroids
In [36]:
%%time
k = 10
heterogeneity_smart = {}
seeds = [0, 20000, 40000, 60000, 80000, 100000, 120000]
for seed in seeds:
initial_centroids = smart_initialize(tf_idf, k, seed)
centroids, cluster_assignment = kmeans(tf_idf, k, initial_centroids, maxiter=400,
record_heterogeneity=None, verbose=False)
# To save time, compute heterogeneity only once in the end
heterogeneity_smart[seed] = compute_heterogeneity(tf_idf, k, centroids, cluster_assignment)
print('seed={0:06d}, heterogeneity={1:.5f}'.format(seed, heterogeneity_smart[seed]))
sys.stdout.flush()
In [37]:
plt.figure(figsize=(8,5))
plt.boxplot([list(heterogeneity.values()), list(heterogeneity_smart.values())], vert=False)
plt.yticks([1, 2], ['k-means', 'k-means++'])
plt.rcParams.update({'font.size': 16})
plt.tight_layout()
In [38]:
def kmeans_multiple_runs(data, k, maxiter, num_runs, seed_list=None, verbose=False):
heterogeneity = {}
min_heterogeneity_achieved = float('inf')
best_seed = None
final_centroids = None
final_cluster_assignment = None
for i in range(num_runs):
# Use UTC time if no seeds are provided
if seed_list is not None:
seed = seed_list[i]
np.random.seed(seed)
else:
seed = int(time.time())
np.random.seed(seed)
# Use k-means++ initialization
# YOUR CODE HERE
initial_centroids = smart_initialize(data, k, seed)
# Run k-means
# YOUR CODE HERE
centroids, cluster_assignment = kmeans(tf_idf, k, initial_centroids, maxiter=400,
record_heterogeneity=None, verbose=False)
# To save time, compute heterogeneity only once in the end
# YOUR CODE HERE
heterogeneity[seed] = compute_heterogeneity(tf_idf, k, centroids, cluster_assignment)
if verbose:
print('seed={0:06d}, heterogeneity={1:.5f}'.format(seed, heterogeneity[seed]))
sys.stdout.flush()
# if current measurement of heterogeneity is lower than previously seen,
# update the minimum record of heterogeneity.
if heterogeneity[seed] < min_heterogeneity_achieved:
min_heterogeneity_achieved = heterogeneity[seed]
best_seed = seed
final_centroids = centroids
final_cluster_assignment = cluster_assignment
# Return the centroids and cluster assignments that minimize heterogeneity.
return final_centroids, final_cluster_assignment
In [39]:
%%time
import numpy as np
def plot_k_vs_heterogeneity(k_values, heterogeneity_values):
plt.figure(figsize=(7,4))
plt.plot(k_values, heterogeneity_values, linewidth=4)
plt.xlabel('K')
plt.ylabel('Heterogeneity')
plt.title('K vs. Heterogeneity')
plt.rcParams.update({'font.size': 16})
plt.tight_layout()
centroids = {}
cluster_assignment = {}
heterogeneity_values = []
k_list = [2, 10, 25, 50, 100]
#seed_list = [0]
# Uncomment the following line to run the plot with all the seeds (it may take about an hour to finish).
seed_list = [0, 20000, 40000, 60000, 80000, 100000, 120000]
for k in k_list:
heterogeneity = []
centroids[k], cluster_assignment[k] = kmeans_multiple_runs(tf_idf, k, maxiter=400,
num_runs=len(seed_list), seed_list=seed_list,
verbose=True)
score = compute_heterogeneity(tf_idf, k, centroids[k], cluster_assignment[k])
heterogeneity_values.append(score)
In [40]:
plot_k_vs_heterogeneity(k_list, heterogeneity_values)
In [69]:
def visualize_document_clusters(wiki, tf_idf, centroids, cluster_assignment, k, map_word_to_index, display_content=True):
'''wiki: original dataframe
tf_idf: data matrix, sparse matrix format
map_index_to_word: SFrame specifying the mapping betweeen words and column indices
display_content: if True, display 8 nearest neighbors of each centroid'''
map_index_to_word = {v:k for k,v in map_word_to_index.items()}
print('==========================================================')
# Visualize each cluster c
for c in range(k):
# Cluster heading
print('Cluster {0:d} '.format(c)),
# Print top 5 words with largest TF-IDF weights in the cluster
idx = centroids[c].argsort()[::-1]
for i in range(5): # Print each word along with the TF-IDF weight
print('{0:s}:{1:.3f}'.format(map_index_to_word[idx[i]], centroids[c][idx[i]])),
print('')
if display_content:
# Compute distances from the centroid to all data points in the cluster,
# and compute nearest neighbors of the centroids within the cluster.
distances = pairwise_distances(tf_idf, centroids[c].reshape(1, -1), metric='euclidean').flatten()
distances[cluster_assignment!=c] = float('inf') # remove non-members from consideration
nearest_neighbors = distances.argsort()
# For 8 nearest neighbors, print the title as well as first 180 characters of text.
# Wrap the text at 80-character mark.
for i in range(8):
text = ' '.join(wiki[nearest_neighbors[i]]['text'].split(None, 25)[0:25])
print('\n* {0:50s} {1:.5f}\n {2:s}\n {3:s}'.format(wiki[nearest_neighbors[i]]['name'],
distances[nearest_neighbors[i]], text[:90], text[90:180] if len(text) > 90 else ''))
print('==========================================================')
In [70]:
'''Notice the extra pairs of parentheses for centroids and cluster_assignment.
The centroid and cluster_assignment are still inside the npz file,
and we need to explicitly indicate when to load them into memory.'''
visualize_document_clusters(wiki, tf_idf, centroids[2], cluster_assignment[2], 2, map_index_to_word)
In [71]:
k = 10
visualize_document_clusters(wiki, tf_idf, centroids[k], cluster_assignment[k], k, map_index_to_word)
In [72]:
np.bincount(cluster_assignment[10])
Out[72]:
In [75]:
visualize_document_clusters(wiki, tf_idf, centroids[25], cluster_assignment[25], 25,
map_index_to_word, display_content=False) # turn off text for brevity
In [76]:
k=100
visualize_document_clusters(wiki, tf_idf, centroids[k], cluster_assignment[k], k,
map_index_to_word, display_content=False)
# turn off text for brevity -- turn it on if you are curious ;)
In [85]:
sum(np.bincount(cluster_assignment[k]) < 236) / k
Out[85]:
In [81]:
k
Out[81]:
In [86]:
sum(np.bincount(cluster_assignment[k]) < 236)
Out[86]:
In [ ]:
In [ ]:
!unzip images.sf.zip
In [2]:
from __future__ import print_function # to conform python 2.x print to python 3.x
import turicreate
import numpy as np
import matplotlib.pyplot as plt
import copy
from scipy.stats import multivariate_normal
%matplotlib inline
In [3]:
def log_sum_exp(Z):
""" Compute log(\sum_i exp(Z_i)) for some array Z."""
return np.max(Z) + np.log(np.sum(np.exp(Z - np.max(Z))))
def loglikelihood(data, weights, means, covs):
""" Compute the loglikelihood of the data for a Gaussian mixture model with the given parameters. """
num_clusters = len(means)
num_dim = len(data[0])
ll = 0
for d in data:
Z = np.zeros(num_clusters)
for k in range(num_clusters):
# Compute (x-mu)^T * Sigma^{-1} * (x-mu)
delta = np.array(d) - means[k]
exponent_term = np.dot(delta.T, np.dot(np.linalg.inv(covs[k]), delta))
# Compute loglikelihood contribution for this data point and this cluster
Z[k] += np.log(weights[k])
Z[k] -= 1/2. * (num_dim * np.log(2*np.pi) + np.log(np.linalg.det(covs[k])) + exponent_term)
# Increment loglikelihood contribution of this data point across all clusters
ll += log_sum_exp(Z)
return ll
In [4]:
def compute_responsibilities(data, weights, means, covariances):
'''E-step: compute responsibilities, given the current parameters'''
num_data = len(data)
num_clusters = len(means)
resp = np.zeros((num_data, num_clusters))
# Update resp matrix so that resp[i,k] is the responsibility of cluster k for data point i.
# Hint: To compute likelihood of seeing data point i given cluster k, use multivariate_normal.pdf.
for i in range(num_data):
for k in range(num_clusters):
# YOUR CODE HERE
resp[i, k] = ...
# Add up responsibilities over each data point and normalize
row_sums = resp.sum(axis=1)[:, np.newaxis]
resp = resp / row_sums
return resp
In [5]:
images = turicreate.SFrame('images.sf')
import array
images['rgb'] = images.pack_columns(['red', 'green', 'blue'])['X4']
# The result will pop out in a separate window
images.explore()
| path | image | folder | red | green | blue | rgb | |
|---|---|---|---|---|---|---|---|
| 0 | /data/coursera/images/sunsets/ANd9GcSN4TPL6_XoTvZeg3-15UhGnWAwjhbxQLjTNiCpWIqMyzq27xIdlg.jpg |  |
sunsets | 0.403223 | 0.254570 | 0.297392 | [0.4032234842076981, 0.2545700932661306, 0.29739183218564663] |
| 1 | /data/coursera/images/sunsets/ANd9GcQeme67tTCcvFbjg3xtvKPls3300iLBXVDEUfy8mx7yWaCAIqEWAw.jpg |  |
sunsets | 0.556835 | 0.246377 | 0.039541 | [0.5568352144946861, 0.24637719918003423, 0.03954077190224097] |
| 2 | /data/coursera/images/sunsets/ANd9GcSAb2GMlHYIvV8eXZuUskgqHA-Oo2LfLjw3FsyeSDF0-5z1rzyk.jpg |  |
sunsets | 0.530449 | 0.114891 | 0.072064 | [0.5304492672627918, 0.11489094324391455, 0.07206392821659215] |
| 3 | /data/coursera/images/sunsets/ANd9GcRs4-CSokZQJFe9vodGC6fK7ouonFopisgxltdHeLmmR85ny4hA.jpg |  |
sunsets | 0.457634 | 0.194312 | 0.616880 | [0.4576339546457987, 0.19431157564148255, 0.6168796125118793] |
| 4 | /data/coursera/images/sunsets/ANd9GcQov2JpsVI0UO50J4NiyhsuxMNw90ffXe6U7PCHVJeQFIUKLZ4qdA.jpg |  |
sunsets | 0.329559 | 0.231696 | 0.202149 | [0.32955887691971036, 0.23169615548161854, 0.20214944771027293] |
| 5 | /data/coursera/images/sunsets/ANd9GcRPQowxaXHVSyfep_8onh-Gjg6P1J5Pux6rEUe3Xuq25aeWwn82.jpg |  |
sunsets | 0.431074 | 0.303505 | 0.270772 | [0.4310738970368823, 0.30350462147887325, 0.27077178821516673] |
| 6 | /data/coursera/images/sunsets/ANd9GcTIIpd0qDRpnGpffSCoHgc6iS4q5SgNDoXtAuXna-1WgB7qSbb-TA.jpg |  |
sunsets | 0.331973 | 0.239819 | 0.153039 | [0.3319727876348366, 0.23981891904330693, 0.15303946769394577] |
| 7 | /data/coursera/images/sunsets/ANd9GcQm3Dvfoztf_kp79I9Lr487O3KOFJeo9q0E2TmbrBr9hz9MgKcB.jpg |  |
sunsets | 0.417399 | 0.401744 | 0.550714 | [0.4173993262543467, 0.4017440542722305, 0.5507135649528068] |
| 8 | /data/coursera/images/sunsets/ANd9GcTCaO2qoZveB_UDbtBh8eit0Z3kThjh-sxKlVMsEM1_T58N7r5J.jpg |  |
sunsets | 0.578920 | 0.314133 | 0.108881 | [0.578919523099851, 0.31413344510680574, 0.10888055762543468] |
| 9 | /data/coursera/images/sunsets/ANd9GcTZ_TW2uDnyBfL_9gkss-9BwWnM82dbFCi_omn-SeA7NkNYcxX6mQ.jpg |  |
sunsets | 0.318551 | 0.319905 | 0.334770 | [0.3185505930203676, 0.31990507016890213, 0.33476954483358173] |
| 10 | /data/coursera/images/sunsets/ANd9GcQBmz9VBm4Okn1pjl7lqfdP47hq0PxOHtnNtjhFOhSoE0doLQzW.jpg |  |
sunsets | 0.575210 | 0.289107 | 0.377153 | [0.5752103484623016, 0.289106677827381, 0.377153087797619] |
| 11 | /data/coursera/images/sunsets/ANd9GcSL1kq5zUHjlrKgr_MU1kVm4k3huzMPR3ifCMgWlswTQIBGPJPOXg.jpg |  |
sunsets | 0.532248 | 0.468598 | 0.389243 | [0.5322484320665674, 0.46859809674615005, 0.38924265710382516] |
| 12 | /data/coursera/images/sunsets/ANd9GcSO1N3EzBmkWq-WzZzT1ZhlKwnrhAbcLtwDUp3BLVMiQqVRDfSp9Q.jpg |  |
sunsets | 0.465509 | 0.206380 | 0.083270 | [0.46550943809869233, 0.20637953552903535, 0.08326970157299575] |
| 13 | /data/coursera/images/sunsets/ANd9GcTfpwOtPZJ8aqbHY1m91H0gSnPB2GZROK4OW5ilTX148q4Ya-4LyA.jpg |  |
sunsets | 0.745278 | 0.257555 | 0.083041 | [0.7452783088454802, 0.25755532778728657, 0.0830412806492059] |
| 14 | /data/coursera/images/sunsets/ANd9GcQCV8_89q_UtP6v7HpojrNvIlDKIFcGVltUayatuPhAAal19B7t.jpg |  |
sunsets | 0.681248 | 0.533516 | 0.325738 | [0.6812476366277913, 0.5335155721350954, 0.32573776084165906] |
| 15 | /data/coursera/images/sunsets/ANd9GcSn7xh8NQ95nLShi_wkshZY2xWcuXdx57sDXnCWIcUrNOAp2U2Isg.jpg |  |
sunsets | 0.411161 | 0.504357 | 0.578829 | [0.41116110181051585, 0.5043566313244048, 0.5788290550595238] |
| 16 | /data/coursera/images/sunsets/ANd9GcRHGdeJnvFvnXkH2PmzkbDzoHzdSbMuO2ks8nqmjtJVCCCF9Xl1FA.jpg |  |
sunsets | 0.439749 | 0.519179 | 0.618196 | [0.43974897540983604, 0.5191793964232488, 0.6181964108296075] |
| 17 | /data/coursera/images/sunsets/ANd9GcT8JlzLyo03Pp97H2B2qQMiFVGQ3tNCNxK3Wq28cUZvMx8A9sCA.jpg |  |
sunsets | 0.553954 | 0.355582 | 0.320426 | [0.553953717251951, 0.35558197463768115, 0.3204255691093327] |
| 18 | /data/coursera/images/sunsets/ANd9GcSzeXthxtaN9plSJFOKddJQI5E4WB4Rzzsu-BD0CY5DdBwh96ngQg.jpg |  |
sunsets | 0.743875 | 0.432845 | 0.450786 | [0.7438753570541481, 0.4328445572528564, 0.45078598484848487] |
| 19 | /data/coursera/images/sunsets/ANd9GcRdsHi_PwhUF3-9lVHHCy5COZoLwJAOYU80b1FWhC6IkwDNMbNB0A.jpg |  |
sunsets | 0.443751 | 0.280791 | 0.291783 | [0.44375077620466963, 0.28079071969696967, 0.29178294212617983] |
| 20 | /data/coursera/images/sunsets/ANd9GcSHnEuQVIA8JxsVb_d09QJOt3bKqK75jZFyJVjp_RNi0YOx8Wv_og.jpg |  |
sunsets | 0.418087 | 0.265995 | 0.210825 | [0.41808657786885245, 0.26599478390461995, 0.21082495032290113] |
| 21 | /data/coursera/images/sunsets/ANd9GcRn1a0FRSODbYAlzwMTd0VfIzbftWztX1-M38i-lwZ3hK4E2AYJ-Q.jpg |  |
sunsets | 0.289492 | 0.141854 | 0.014024 | [0.2894921103013834, 0.14185438796936758, 0.01402436388339921] |
| 22 | /data/coursera/images/sunsets/ANd9GcRZY5x07-GUlyC9JOYmwnkwXa4LbP9jKiquDZX-7eNfCgQPusNfUg.jpg |  |
sunsets | 0.606827 | 0.413317 | 0.414274 | [0.60682666015625, 0.4133173828125, 0.414274169921875] |
| 23 | /data/coursera/images/sunsets/ANd9GcQY71eLp6Z3qb-m8QiW-urhxVeKhGpC_JFqgWDwtuM27BeMyW-w.jpg |  |
sunsets | 0.659578 | 0.451834 | 0.480988 | [0.6595779854910714, 0.4518341548859127, 0.480988033234127] |
| 24 | /data/coursera/images/sunsets/ANd9GcQYXnPhvKwOXyzG0U0XULdrnt0enuJADhuDuo1UWMFA2jV4lUu6.jpg |  |
sunsets | 0.476831 | 0.267148 | 0.357970 | [0.47683067871336315, 0.2671477583209141, 0.3579699919274714] |
| 25 | /data/coursera/images/sunsets/ANd9GcQj3sU9Gf4MfBQaOnrpAFByyMZqWuhONdRfC-TNPTofV8S21BgtYA.jpg |  |
sunsets | 0.672485 | 0.480775 | 0.136721 | [0.6724851377110208, 0.4807750630822981, 0.13672116068940118] |
| 26 | /data/coursera/images/sunsets/ANd9GcQQu3ojtg_WSsdPM6X4iNaoejKW58tJmz8gEwBegOmmTAwQ8CXmaA.jpg |  |
sunsets | 0.465263 | 0.240950 | 0.081812 | [0.4652632843501984, 0.24094974578373016, 0.08181206597222222] |
| 27 | /data/coursera/images/sunsets/ANd9GcRiVGX2-ar-hH2D8GAXCxLT_uW2pQSTLut9T-sF9ITJ6y9u3S0Vcw.jpg |  |
sunsets | 0.381877 | 0.444458 | 0.453180 | [0.3818773561211904, 0.44445762898364766, 0.45318032843956396] |
| 28 | /data/coursera/images/sunsets/ANd9GcQtFjZzNBLOpzTLyt7eWW7hTIvZQni8UbHZfJjiRXohldnMmgsN.jpg |  |
sunsets | 0.549877 | 0.391405 | 0.446488 | [0.5498773596621958, 0.3914049304520616, 0.4464880619721808] |
| 29 | /data/coursera/images/sunsets/ANd9GcS6UV30PMi6vrUMsUt80oW_24U0V3ss_z-qeHj1G4RDjEOntSwW.jpg |  |
sunsets | 0.722454 | 0.200460 | 0.303308 | [0.7224544304521753, 0.20045983137961232, 0.3033080369581658] |
| 30 | /data/coursera/images/sunsets/ANd9GcTi_xqhKxHngomZxpQOSqzLUMhy1S1qfPEWTcbN3aXL7HK7SMQNrw.jpg |  |
sunsets | 0.773048 | 0.570237 | 0.313390 | [0.7730481557377049, 0.5702365095628416, 0.3133903843765524] |
| 31 | /data/coursera/images/sunsets/ANd9GcR_09vZLKNYUkOFtc1y_rkzMOpO0e6VS0QaSqDNnz0O1i5QVi3t.jpg |  |
sunsets | 0.695401 | 0.346390 | 0.075317 | [0.6954009331597222, 0.3463899739583333, 0.07531722780257936] |
| 32 | /data/coursera/images/sunsets/ANd9GcRdZd6pv8D7DBPRZAO9tXy-HAyGSKcZg6Ebbs7l3i5xCfgu-YSC.jpg |  |
sunsets | 0.495876 | 0.155497 | 0.065678 | [0.4958762436046743, 0.15549716502926422, 0.06567814403666986] |
| 33 | /data/coursera/images/sunsets/ANd9GcR04DFOem_nOZbkjwX_3BDAXtiPmF3mag5mwdlwAnZH5ceFmGoh.jpg |  |
sunsets | 0.400289 | 0.356060 | 0.357375 | [0.40028882575757574, 0.35605975223546943, 0.35737549677098857] |
| 34 | /data/coursera/images/sunsets/ANd9GcQMLiOIwamPqz4FKFXyUH_hf2bstJvVft__uyvyFNzb5b9lyA5A.jpg |  |
sunsets | 0.542837 | 0.356715 | 0.318294 | [0.5428369140625, 0.356715339781746, 0.3182941158234127] |
| 35 | /data/coursera/images/sunsets/ANd9GcSDEFQ_-wH3RCUKLylXnAIruTB9FArzxn68LD0_yuTQApnNBC20.jpg |  |
sunsets | 0.619282 | 0.571417 | 0.569110 | [0.6192816225782414, 0.5714173497267759, 0.5691100037257825] |
| 36 | /data/coursera/images/sunsets/ANd9GcScNyrHeWLvH7am0TpltiEEJ9CosecsRC1wSlVBTqz9iswJOLjJDw.jpg |  |
sunsets | 0.515521 | 0.522777 | 0.534728 | [0.5155214542970691, 0.5227773379284649, 0.5347281731246896] |
| 37 | /data/coursera/images/sunsets/ANd9GcTWXsYFNN0fYA2QetcrF1Irx330m11oMNQOS8Qai34ArGDUROO0.jpg |  |
sunsets | 0.442885 | 0.157788 | 0.054802 | [0.4428845021081349, 0.1577876984126984, 0.05480174231150794] |
| 38 | /data/coursera/images/sunsets/ANd9GcSe81J9HiRw_02Hlgf3Wg5cjEPKPJLC7zYU5ofEwPkuJ4oqRRS9.jpg |  |
sunsets | 0.412819 | 0.260080 | 0.291720 | [0.41281893115401447, 0.2600799558118485, 0.29171950843478955] |
| 39 | /data/coursera/images/sunsets/ANd9GcSdNeOBQTNbqPuEk5U-TVs8Dln66ujCasf11N1ieUHiEPV2TRVBpw.jpg |  |
sunsets | 0.508744 | 0.310266 | 0.437166 | [0.5087436351217088, 0.3102658500993542, 0.4371663096125186] |
| 40 | /data/coursera/images/sunsets/ANd9GcS59y5m6MmNqym00PBKNjDPqmpMy8obOLBuOcUAVTanZZOJxt9W.jpg |  |
sunsets | 0.509290 | 0.202590 | 0.333762 | [0.5092896174863388, 0.20259035022354693, 0.3337617206905117] |
| 41 | /data/coursera/images/sunsets/ANd9GcScj81Z5hZTG1IOjN6e7HjbAm8l0y8s8biNFDUdn2cwthi9UBNl1Q.jpg |  |
sunsets | 0.591565 | 0.187007 | 0.042252 | [0.5915654591390149, 0.18700748363770192, 0.042252044665592424] |
| 42 | /data/coursera/images/sunsets/ANd9GcRk42vFH08TygmvldqZ1IEDPcHIxR_Ilcn88iTXx9PEhjGxfAre.jpg |  |
sunsets | 0.543546 | 0.248770 | 0.255513 | [0.5435462017337074, 0.24876971930452774, 0.25551326141879527] |
| 43 | /data/coursera/images/sunsets/ANd9GcTpv1GoGEC6n0DBYdJavt99lKiEXEv9SE4DoedkF-LGBncy8v3G.jpg |  |
sunsets | 0.471613 | 0.236832 | 0.208774 | [0.4716129533035271, 0.2368323087431694, 0.2087737518628912] |
| 44 | /data/coursera/images/sunsets/ANd9GcT64JvuOrtwaH6zF_kWoNhfV3lXS85A9fANobbt0p6SrDMThKFClg.jpg |  |
sunsets | 0.438263 | 0.322950 | 0.249257 | [0.4382634879776217, 0.32294991233682496, 0.2492565834176487] |
| 45 | /data/coursera/images/sunsets/ANd9GcTHndsUHJHc03mGWbCfZeXjOkDIY2kLmyOSGDAQamV7aQJ-Nqy0jQ.jpg |  |
sunsets | 0.585562 | 0.376926 | 0.213066 | [0.5855616657459388, 0.37692598530319316, 0.21306559439202102] |
| 46 | /data/coursera/images/sunsets/ANd9GcQWa02JP_-LQNysjkhufzln-sL3QPj_ka__scvNWd0ibnt1EPSb6w.jpg |  |
sunsets | 0.483128 | 0.407718 | 0.355157 | [0.48312779433681075, 0.4077181135121709, 0.3551571814456036] |
| 47 | /data/coursera/images/sunsets/ANd9GcSOBwVrU-cVKExiQEgNWziiFSF5MU8MsWXdcC_jAgTpwsBiRpWU.jpg |  |
sunsets | 0.364768 | 0.101154 | 0.115004 | [0.36476823092883015, 0.10115419635393862, 0.11500386846714167] |
| 48 | /data/coursera/images/sunsets/ANd9GcQk-WeL6N3tb6a79QiUQs-dDp-PI_Ye7dHviKjJAoj-IvN69rQiDg.jpg |  |
sunsets | 0.252793 | 0.281998 | 0.277426 | [0.25279276515377574, 0.2819982549783162, 0.2774258101575555] |
| 49 | /data/coursera/images/sunsets/ANd9GcRuM9jj0NZmoAvB5jMOtCg01f-Ng27IjKjCMX_1cqa9rKk4gqOt.jpg |  |
sunsets | 0.413875 | 0.441579 | 0.435464 | [0.4138746741034112, 0.44157938256279106, 0.43546423471520124] |
| 50 | /data/coursera/images/sunsets/ANd9GcR2xvU0J9rTDVrcFAgtfar2iSk0ENuCgsH7KRmN9Uf6BR-S-GZujw.jpg |  |
sunsets | 0.443760 | 0.267036 | 0.073878 | [0.44376001304023843, 0.2670364505712866, 0.07387815139095877] |
| 51 | /data/coursera/images/sunsets/ANd9GcRctsS1SN9Kvcp1j5iMjih4jv9JrbL5xCHMSWSW0na76Q7bQovw.jpg |  |
sunsets | 0.452440 | 0.310596 | 0.257824 | [0.45243993302949487, 0.3105960082643395, 0.2578241613760299] |
| 52 | /data/coursera/images/sunsets/ANd9GcT0shDIioVJO9-VH7VvwB-h6raUNYztF8e6BTmCuVG-v2MvEsz_Xg.jpg |  |
sunsets | 0.579431 | 0.378367 | 0.251075 | [0.5794314236111111, 0.3783672030009921, 0.25107530381944443] |
| 53 | /data/coursera/images/sunsets/ANd9GcRRf0d8e7GZfwKAl8BiMqwj74Z24gerEfmTML_Kz8s2_PyMMGXADg.jpg |  |
sunsets | 0.466466 | 0.160419 | 0.340548 | [0.4664658625186289, 0.16041860717834078, 0.3405477676353701] |
| 54 | /data/coursera/images/sunsets/ANd9GcTohz4jLZMHwLOnwnUFl5wLgEPd89_DincOcBDGmKMNx8Und-JTgQ.jpg |  |
sunsets | 0.514728 | 0.514224 | 0.541076 | [0.5147275555357567, 0.5142236130909955, 0.5410763362982102] |
| 55 | /data/coursera/images/sunsets/ANd9GcSfgH0w3HIR0J7yhFdTrWai5f3K2dM7gJJWp79vl6gylwCwXYZXng.jpg |  |
sunsets | 0.384980 | 0.363514 | 0.285966 | [0.384980101320447, 0.3635140589577616, 0.2859663816718371] |
| 56 | /data/coursera/images/sunsets/ANd9GcS97_PhVMp6bDW3-VBEv0_dRP1804uXcZa4oPGmc2MGHKO23DkKAg.jpg |  |
sunsets | 0.671842 | 0.565891 | 0.583955 | [0.6718417796467703, 0.5658906392152209, 0.5839545539387286] |
| 57 | /data/coursera/images/sunsets/ANd9GcSlo6r4Qa6Xnfj6t0gXLugxCbGQEz8dhq8kWFchwLZTlrR_JBJGFQ.jpg |  |
sunsets | 0.630192 | 0.382246 | 0.163806 | [0.6301920572916667, 0.3822457837301587, 0.16380564856150795] |
| 58 | /data/coursera/images/sunsets/ANd9GcS3jzoIsv2p_HUHs73YnXUPLV7Mt2TPFz1y0sMga1R78Do0300w.jpg |  |
sunsets | 0.443019 | 0.272771 | 0.080626 | [0.44301920983837945, 0.2727710698169095, 0.08062640399180963] |
| 59 | /data/coursera/images/sunsets/ANd9GcTIozvbWNOTPiQ6c4VRIZ7t9u-I9EXD1l1eAAaO1iCTlG4nrdIBlA.jpg |  |
sunsets | 0.377749 | 0.126360 | 0.092872 | [0.37774948708620415, 0.12636039711530847, 0.09287204502257068] |
| 60 | /data/coursera/images/sunsets/ANd9GcTk_Y-Gwpbse6DT0TDV4dYWLZaybJkGT8F8az_xmuZ-Vc83Y6Oz4A.jpg |  |
sunsets | 0.491294 | 0.335929 | 0.239821 | [0.4912938524508634, 0.3359291851923689, 0.23982096587292115] |
| 61 | /data/coursera/images/sunsets/ANd9GcQZ6ZKxfRN7ixcj1cwdMHJP1cSKmtpa090W8cikckkYiZAeFl8Q.jpg |  |
sunsets | 0.379466 | 0.200094 | 0.238525 | [0.3794658358134921, 0.20009362599206348, 0.2385254681299603] |
| 62 | /data/coursera/images/sunsets/ANd9GcRScYWqbMMB9bj6yLEDc9sVBhKuLpkGX6XDlXUIDvVKeARTMG5V.jpg |  |
sunsets | 0.623014 | 0.370704 | 0.110402 | [0.6230137698708396, 0.37070417287630403, 0.11040246212121212] |
| 63 | /data/coursera/images/sunsets/ANd9GcQa35xTzYNjAQNDmtLwFeDlzmWaIo8FCdO2s5vDLJg0UEMyb7DN.jpg |  |
sunsets | 0.335102 | 0.371289 | 0.485207 | [0.3351016739744352, 0.3712885980231867, 0.48520672314704716] |
| 64 | /data/coursera/images/sunsets/ANd9GcT1lcYtmDTsCb-WwpofjsIne3AWkAd8WDz1328UOGizlxjdKIFq.jpg |  |
sunsets | 0.309545 | 0.378142 | 0.335398 | [0.3095446783407849, 0.3781423093641331, 0.33539842585692997] |
| 65 | /data/coursera/images/sunsets/ANd9GcSKElB-04bh4dpM1nCtbyvAfZIeEvdx0QJWr0eQ-zOG5Jxopx-d.jpg |  |
sunsets | 0.585963 | 0.267481 | 0.175817 | [0.5859629995422522, 0.2674805581538829, 0.17581724478067906] |
| 66 | /data/coursera/images/sunsets/ANd9GcSgx8rNDq8i6vN1LZJFdDBumHD6XgQyyZOhzCr2cl9Rcq59bZPd.jpg |  |
sunsets | 0.286388 | 0.279029 | 0.330032 | [0.2863882420516642, 0.27902857985593643, 0.3300316691505216] |
| 67 | /data/coursera/images/sunsets/ANd9GcQaMTGizr08BdDihGFW22sxHyZUFZazT0-DhlYUjZLceKYUv87Jvw.jpg |  |
sunsets | 0.464538 | 0.199905 | 0.409143 | [0.4645377526383911, 0.19990510503783354, 0.4091432851702509] |
| 68 | /data/coursera/images/sunsets/ANd9GcT23Os5u_Nm5DMZ6yW0YsYDIMsx720OrxuzF_oGvPbGRo3nXc1u.jpg |  |
sunsets | 0.294156 | 0.215644 | 0.214431 | [0.29415622585581835, 0.21564414509786442, 0.2144307452008796] |
| 69 | /data/coursera/images/sunsets/ANd9GcTHq57PL5KvIcXl6_vjFHO4s-Vyh6rIPNMx9gZxdz1aZleh6Ssa.jpg |  |
sunsets | 0.415247 | 0.535832 | 0.502496 | [0.41524677579365077, 0.5358323257688492, 0.5024958922371032] |
| 70 | /data/coursera/images/sunsets/ANd9GcRBaacedfTfxlzjyPY2HK0HOJKXVMWRMzlbYt1QcyI1xoWXkpXeMQ.jpg |  |
sunsets | 0.531277 | 0.425165 | 0.297173 | [0.5312770492311508, 0.4251651630704365, 0.29717347160218255] |
| 71 | /data/coursera/images/sunsets/ANd9GcSYhWwY-sXem8YVU3PknNC4UCRRMuLmgOlfSptznw2Hk9CY1AOT.jpg |  |
sunsets | 0.621522 | 0.450985 | 0.521145 | [0.6215222925981123, 0.4509850813462494, 0.5211454452309985] |
| 72 | /data/coursera/images/sunsets/ANd9GcQkM2nQvq8FPmcA-T6DchVkGohw-sp6KbqMaTtFds61p_0-rLgzQQ.jpg |  |
sunsets | 0.290299 | 0.184462 | 0.207466 | [0.2902993936975375, 0.1844621163523889, 0.20746628786748617] |
| 73 | /data/coursera/images/sunsets/ANd9GcR-YX1KgguWCg0MNxzSDxogqGBxOVyTFN2SseQWX-8gK62MN83xVg.jpg |  |
sunsets | 0.554009 | 0.295853 | 0.196315 | [0.5540094669408806, 0.29585288469153403, 0.19631504848241862] |
| 74 | /data/coursera/images/sunsets/ANd9GcTX5X9PZ2f1sDudW9vPLXoKL5ckSNrJS7fu0uui1Y9EhA7v-vC2.jpg |  |
sunsets | 0.435353 | 0.309032 | 0.278687 | [0.4353533451140873, 0.30903173053075395, 0.27868714347718254] |
| 75 | /data/coursera/images/sunsets/ANd9GcQV2VuOnacUOwUZ8vqvSL4uaLLntgTRqnPo8bjV-npkgItZXaRK.jpg |  |
sunsets | 0.402675 | 0.356356 | 0.346943 | [0.4026751277361379, 0.3563563328793813, 0.3469434811079743] |
| 76 | /data/coursera/images/sunsets/ANd9GcQPtkSzAaqQJIoHlwwmjZeFef_RukkLkxfaJzTkNtMXfuYYU3hl.jpg |  |
sunsets | 0.416591 | 0.188316 | 0.101859 | [0.4165913748137109, 0.18831610158966716, 0.10185862208147044] |
| 77 | /data/coursera/images/sunsets/ANd9GcRhL1IJeKEURBZpbP2shUQ41xrIYryPtDFsg2yKno4FawMshPGRkQ.jpg |  |
sunsets | 0.608117 | 0.426154 | 0.395832 | [0.6081169274714356, 0.4261540611028316, 0.3958324795081967] |
| 78 | /data/coursera/images/sunsets/ANd9GcRhF0P5E1Y-F8FHhHkHJa0ec1y4fa78ywBE7Mu68NJWXMP9RbqXRg.jpg |  |
sunsets | 0.531097 | 0.216416 | 0.095466 | [0.5310973576288333, 0.21641643132364025, 0.095465948198913] |
| 79 | /data/coursera/images/sunsets/ANd9GcTPV9Ybox40SXw4rpACDUZ1EBynvRVJnunalJNcDijCI0FavvPy.jpg |  |
sunsets | 0.408670 | 0.375327 | 0.345399 | [0.4086701242387454, 0.3753272182113999, 0.34539860760060503] |
| 80 | /data/coursera/images/sunsets/ANd9GcSzRSJPYL2agiGFcgon1F-ONbwMbCQh2RtxPODhDybW7hwSAxJA.jpg |  |
sunsets | 0.197723 | 0.182190 | 0.178018 | [0.19772340682629388, 0.18219019556811422, 0.17801810678167757] |
| 81 | /data/coursera/images/sunsets/ANd9GcQyjcRGJits5hRypGoCs9BLBScDCm34YsPINciwlNnDod7lWuSJ.jpg |  |
sunsets | 0.376630 | 0.168168 | 0.465189 | [0.3766297022956951, 0.16816783913225114, 0.46518939733030956] |
| 82 | /data/coursera/images/sunsets/ANd9GcRk05mYkqARwt87BQNGJ20BNbd4fb_eaAFd1_hval70sa7P_KomMQ.jpg |  |
sunsets | 0.300981 | 0.135470 | 0.257158 | [0.30098057935916545, 0.13547014716840536, 0.25715831470442124] |
| 83 | /data/coursera/images/sunsets/ANd9GcQXlNshRcQ7WX_PrDXQw9esYY8MKy7Wmacg39HEEDp4N2FSsUWw.jpg |  |
sunsets | 0.648614 | 0.450321 | 0.301235 | [0.6486141338045635, 0.45032149057539683, 0.3012353515625] |
| 84 | /data/coursera/images/sunsets/ANd9GcQ-6pQo9fg5S5QMcV8N8J54yYqrubYCWw-bq4-Da1Ku1hkEJEO3.jpg |  |
sunsets | 0.827147 | 0.653187 | 0.432993 | [0.8271469821162444, 0.6531869411326379, 0.43299304520615994] |
| 85 | /data/coursera/images/sunsets/ANd9GcQgCTRJROBCiF-kcOxAmwiKnb0UkY3aqtQOCPvuvoLOVwKXZqkw9A.jpg |  |
sunsets | 0.383679 | 0.483137 | 0.619628 | [0.3836793954670547, 0.48313703089114596, 0.6196284035449829] |
| 86 | /data/coursera/images/sunsets/ANd9GcQVSV4PiEMenWys7kH1QVPmpbV43zbfTdu9-hWbVfpeAxbq8XVQEw.jpg |  |
sunsets | 0.241059 | 0.144941 | 0.100495 | [0.24105870863970588, 0.14494128204491255, 0.10049464676073132] |
| 87 | /data/coursera/images/sunsets/ANd9GcR1ZoGvVPetGeDhINF9g8ycjdvPYWiGSduJNTFLakwsET9Y9CJR.jpg |  |
sunsets | 0.461697 | 0.233015 | 0.056374 | [0.46169728112599207, 0.23301478794642858, 0.05637369791666667] |
| 88 | /data/coursera/images/sunsets/ANd9GcS5RMMQsu9ognNCUVV-eROtdtRFWB_2r0NcIS5Z9Qee7v9dha2n.jpg |  |
sunsets | 0.603878 | 0.461333 | 0.384608 | [0.603877636304115, 0.4613328371145986, 0.3846076761586842] |
| 89 | /data/coursera/images/sunsets/ANd9GcRmGhuHn7UQfIpL0Q8mdNb_L1pNH1NgSIKnT-3ypcTGvAfvlB7C.jpg |  |
sunsets | 0.472948 | 0.096230 | 0.102576 | [0.4729475109300477, 0.09623029796800477, 0.10257640165441176] |
| 90 | /data/coursera/images/sunsets/ANd9GcQBQJncxQKcm335M26pQvhSqS9tPq9BMQFzrn8tOXKhpOrqYi071w.jpg |  |
sunsets | 0.371514 | 0.412192 | 0.477004 | [0.3715135214853453, 0.41219184674615, 0.47700431569796325] |
| 91 | /data/coursera/images/sunsets/ANd9GcSB33ZBKEQhT7tZCua7rhuHepu2i0tgPIKYpDsuF8rJOHs-9b4MBA.jpg |  |
sunsets | 0.459873 | 0.198164 | 0.530288 | [0.4598728160808069, 0.19816370193264105, 0.5302883575286464] |
| 92 | /data/coursera/images/sunsets/ANd9GcQtEAQ5WEY6aiaaPCPHFZk4z0NGd6oZ5Te_nZHDckoqsLlr4QKz.jpg |  |
sunsets | 0.527999 | 0.388483 | 0.309215 | [0.5279985812891027, 0.388483089931298, 0.3092145520264117] |
| 93 | /data/coursera/images/sunsets/ANd9GcTAC2Eb-Aq_weA9-P17L26UZgQ0wZoatyJFQIDqKNlSN3Zph9jG.jpg |  |
sunsets | 0.628384 | 0.386768 | 0.207165 | [0.6283844375397868, 0.38676809488640884, 0.20716475317299277] |
| 94 | /data/coursera/images/sunsets/ANd9GcRkvq_jSrRFA3ZIDA2Nd_zP9VgetqlJyACwdyF8UdIVRmoPTreFZg.jpg |  |
sunsets | 0.560472 | 0.196484 | 0.035950 | [0.5604716590750697, 0.19648390830346477, 0.03595010080645161] |
| 95 | /data/coursera/images/sunsets/ANd9GcRHWdFeoQcQT5A4iWGdiNyKw9_WLEn6Yvjz0yjh2-4tuw6Yd9Bh.jpg |  |
sunsets | 0.652187 | 0.355665 | 0.144637 | [0.6521867101361303, 0.35566536079986466, 0.14463682432432431] |
| 96 | /data/coursera/images/sunsets/ANd9GcSo6l5a9PEDX1fWvW1TkwLXT6RCJMKaUCv8i-r-UAAyr06WHeWsjA.jpg |  |
sunsets | 0.453424 | 0.249679 | 0.444943 | [0.4534243143066173, 0.24967914501712057, 0.44494280911570316] |
| 97 | /data/coursera/images/sunsets/ANd9GcSRb3OXTt69DIbG0ZDr9WhjwE9LBrZsgE5eqcOzlzJ0-0zDoQpw.jpg |  |
sunsets | 0.194003 | 0.198174 | 0.418481 | [0.19400325298422977, 0.1981743006177008, 0.41848088864006516] |
| 98 | /data/coursera/images/sunsets/ANd9GcRZ6t5IILMj0_bEoTaT8TOnR-0nRMvsdBab7KjEIqwokBxRH23BJQ.jpg |  |
sunsets | 0.512699 | 0.322139 | 0.394569 | [0.5126989552331349, 0.32213898189484125, 0.39456876240079364] |
| 99 | /data/coursera/images/sunsets/ANd9GcRD94TAbd4Pw_uLY2nleDbKnjOAoRrDpt5VxTma-XF0nZcykJJ5pg.jpg |  |
sunsets | 0.604095 | 0.414971 | 0.371276 | [0.6040952731311945, 0.41497092119273177, 0.3712759784360944] |
In [6]:
def log_sum_exp(Z):
""" Compute log(\sum_i exp(Z_i)) for some array Z."""
return np.max(Z) + np.log(np.sum(np.exp(Z - np.max(Z))))
def loglikelihood(data, weights, means, covs):
""" Compute the loglikelihood of the data for a Gaussian mixture model with the given parameters. """
num_clusters = len(means)
num_dim = len(data[0])
ll = 0
for d in data:
Z = np.zeros(num_clusters)
for k in range(num_clusters):
# Compute (x-mu)^T * Sigma^{-1} * (x-mu)
delta = np.array(d) - means[k]
exponent_term = np.dot(delta.T, np.dot(np.linalg.inv(covs[k]), delta))
# Compute loglikelihood contribution for this data point and this cluster
Z[k] += np.log(weights[k])
Z[k] -= 1/2. * (num_dim * np.log(2*np.pi) + np.log(np.linalg.det(covs[k])) + exponent_term)
# Increment loglikelihood contribution of this data point across all clusters
ll += log_sum_exp(Z)
return ll
In [7]:
import numpy as np
from scipy.stats import multivariate_normal
def compute_responsibilities(data, weights, means, covariances):
'''E-step: compute responsibilities, given the current parameters'''
num_data = len(data)
num_clusters = len(means)
resp = np.zeros((num_data, num_clusters))
# Update resp matrix so that resp[i,k] is the responsibility of cluster k for data point i.
# Hint: To compute likelihood of seeing data point i given cluster k, use multivariate_normal.pdf.
for i in range(num_data):
for k in range(num_clusters):
# YOUR CODE HERE
resp[i, k] = weights[k]*multivariate_normal.pdf(data[i], means[k], covariances[k])
# Add up responsibilities over each data point and normalize
row_sums = resp.sum(axis=1)[:, np.newaxis]
resp = resp / row_sums
return resp
In [8]:
def compute_soft_counts(resp):
# Compute the total responsibility assigned to each cluster, which will be useful when
# implementing M-steps below. In the lectures this is called N^{soft}
counts = np.sum(resp, axis=0)
return counts
In [9]:
def compute_weights(counts):
num_clusters = len(counts)
#weights = [0.] * num_clusters
#for k in range(num_clusters):
# Update the weight for cluster k using the M-step update rule for the cluster weight, \hat{\pi}_k.
# HINT: compute # of data points by summing soft counts.
# YOUR CODE HERE
#weights[k] = compute_soft_counts(resp[:,k]) / len(resp)
weights = counts / sum(counts)
return weights
In [10]:
def compute_means(data, resp, counts):
num_clusters = len(counts)
num_data = len(data)
means = [np.zeros(len(data[0]))] * num_clusters
for k in range(num_clusters):
# Update means for cluster k using the M-step update rule for the mean variables.
# This will assign the variable means[k] to be our estimate for \hat{\mu}_k.
weighted_sum = 0.
for i in range(num_data):
# YOUR CODE HERE
weighted_sum += np.sum(resp[i,k], axis=0) * data[i]
# YOUR CODE HERE
means[k] = weighted_sum / compute_soft_counts(resp[:,k])
return means
In [11]:
def compute_covariances(data, resp, counts, means):
num_clusters = len(counts)
num_dim = len(data[0])
num_data = len(data)
covariances = [np.zeros((num_dim,num_dim))] * num_clusters
for k in range(num_clusters):
# Update covariances for cluster k using the M-step update rule for covariance variables.
# This will assign the variable covariances[k] to be the estimate for \hat{\Sigma}_k.
weighted_sum = np.zeros((num_dim, num_dim))
for i in range(num_data):
# YOUR CODE HERE (Hint: Use np.outer on the data[i] and this cluster's mean)
weighted_sum += np.sum(resp[i,k], axis=0) * np.outer(data[i] - means[k], data[i] - means[k])
# YOUR CODE HERE
covariances[k] = weighted_sum / compute_soft_counts(resp[:,k])
return covariances
In [12]:
# SOLUTION
def EM(data, init_means, init_covariances, init_weights, maxiter=1000, thresh=1e-4):
# Make copies of initial parameters, which we will update during each iteration
means = init_means[:]
covariances = init_covariances[:]
weights = init_weights[:]
# Infer dimensions of dataset and the number of clusters
num_data = len(data)
num_dim = len(data[0])
num_clusters = len(means)
# Initialize some useful variables
resp = np.zeros((num_data, num_clusters))
ll = loglikelihood(data, weights, means, covariances)
ll_trace = [ll]
for it in range(maxiter):
if it % 5 == 0:
print("Iteration %s" % it)
# E-step: compute responsibilities
resp = compute_responsibilities(data, weights, means, covariances)
# M-step
# Compute the total responsibility assigned to each cluster, which will be useful when
# implementing M-steps below. In the lectures this is called N^{soft}
counts = compute_soft_counts(resp)
# Update the weight for cluster k using the M-step update rule for the cluster weight, \hat{\pi}_k.
# YOUR CODE HERE
weights = compute_weights(counts)
# Update means for cluster k using the M-step update rule for the mean variables.
# This will assign the variable means[k] to be our estimate for \hat{\mu}_k.
# YOUR CODE HERE
means = compute_means(data, resp, counts)
# Update covariances for cluster k using the M-step update rule for covariance variables.
# This will assign the variable covariances[k] to be the estimate for \hat{\Sigma}_k.
# YOUR CODE HERE
covariances = compute_covariances(data, resp, counts, means)
# Compute the loglikelihood at this iteration
# YOUR CODE HERE
ll_latest = loglikelihood(data, weights, means, covariances)
ll_trace.append(ll_latest)
# Check for convergence in log-likelihood and store
if (ll_latest - ll) < thresh and ll_latest > -np.inf:
break
ll = ll_latest
if it % 5 != 0:
print("Iteration %s" % it)
out = {'weights': weights, 'means': means, 'covs': covariances, 'loglik': ll_trace, 'resp': resp}
return out
In [13]:
np.random.seed(1)
# Initalize parameters
init_means = [images['rgb'][x] for x in np.random.choice(len(images), 4, replace=False)]
cov = np.diag([images['red'].var(), images['green'].var(), images['blue'].var()])
init_covariances = [cov, cov, cov, cov]
init_weights = [1/4., 1/4., 1/4., 1/4.]
# Convert rgb data to numpy arrays
img_data = [np.array(i) for i in images['rgb']]
# Run our EM algorithm on the image data using the above initializations.
# This should converge in about 125 iterations
out = EM(img_data, init_means, init_covariances, init_weights)
In [14]:
ll = out['loglik']
plt.plot(range(len(ll)),ll,linewidth=4)
plt.xlabel('Iteration')
plt.ylabel('Log-likelihood')
plt.rcParams.update({'font.size':16})
plt.tight_layout()
In [15]:
plt.figure()
plt.plot(range(3,len(ll)),ll[3:],linewidth=4)
plt.xlabel('Iteration')
plt.ylabel('Log-likelihood')
plt.rcParams.update({'font.size':16})
plt.tight_layout()
In [16]:
import colorsys
def plot_responsibilities_in_RB(img, resp, title):
N, K = resp.shape
HSV_tuples = [(x*1.0/K, 0.5, 0.9) for x in range(K)]
RGB_tuples = list(map(lambda x: colorsys.hsv_to_rgb(*x), HSV_tuples))
R = img['red']
B = img['blue']
resp_by_img_int = [[resp[n][k] for k in range(K)] for n in range(N)]
cols = [tuple(np.dot(resp_by_img_int[n], np.array(RGB_tuples))) for n in range(N)]
plt.figure()
for n in range(len(R)):
plt.plot(R[n], B[n], 'o', c=cols[n])
plt.title(title)
plt.xlabel('R value')
plt.ylabel('B value')
plt.rcParams.update({'font.size':16})
plt.tight_layout()
In [17]:
N, K = out['resp'].shape
random_resp = np.random.dirichlet(np.ones(K), N)
plot_responsibilities_in_RB(images, random_resp, 'Random responsibilities')
In [18]:
out = EM(img_data, init_means, init_covariances, init_weights, maxiter=1)
plot_responsibilities_in_RB(images, out['resp'], 'After 1 iteration')
In [19]:
out = EM(img_data, init_means, init_covariances, init_weights, maxiter=20)
plot_responsibilities_in_RB(images, out['resp'], 'After 20 iterations')
In [20]:
loglikelihood([img_data[0]], out['weights'], out['means'], out['covs'])
Out[20]:
In [21]:
for k in range(4):
print(out['weights'][k]*multivariate_normal.pdf(img_data[0], out['means'][k], out['covs'][k]))
In [22]:
weights = out['weights']
means = out['means']
covariances = out['covs']
rgb = images['rgb']
N = len(images) # number of images
K = len(means) # number of clusters
assignments = [0]*N
probs = [0]*N
for i in range(N):
# Compute the score of data point i under each Gaussian component:
p = np.zeros(K)
for k in range(K):
p[k] = weights[k]*multivariate_normal.pdf(rgb[i], mean=means[k], cov=covariances[k])
# Compute assignments of each data point to a given cluster based on the above scores:
assignments[i] = np.argmax(p)
# For data point i, store the corresponding score under this cluster assignment:
probs[i] = np.max(p)
assignments = turicreate.SFrame({'assignments':assignments, 'probs':probs, 'image': images['image']})
In [24]:
def get_top_images(assignments, cluster, k=5):
# YOUR CODE HERE
images_in_cluster = assignments[assignments['assignments'] == cluster]
top_images = images_in_cluster.topk('probs', k)
return top_images['image']
In [25]:
# Images will appear in a separate window
for component_id in range(4):
get_top_images(assignments, component_id).explore()
| SArray | |
|---|---|
| 0 |  |
| 1 |  |
| 2 |  |
| 3 |  |
| 4 |  |
| SArray | |
|---|---|
| 0 |  |
| 1 | |
| 2 |  |
| 3 |  |
| 4 |  |
| SArray | |
|---|---|
| 0 |  |
| 1 |  |
| 2 |  |
| 3 |  |
| 4 |  |
| SArray | |
|---|---|
| 0 |  |
| 1 |  |
| 2 | |
| 3 |  |
| 4 |  |
In [ ]:
In [2]:
from __future__ import print_function # to conform python 2.x print to python 3.x
import turicreate
from em_utilities import *
In [3]:
wiki = turicreate.SFrame('people_wiki.sframe/').head(5000)
wiki['tf_idf'] = turicreate.text_analytics.tf_idf(wiki['text'])
In [4]:
wiki = wiki.add_row_number()
tf_idf, map_word_to_index = sframe_to_scipy(wiki, 'tf_idf')
map_index_to_word = dict([[map_word_to_index[i], i] for i in map_word_to_index.keys()])
In [5]:
%%time
tf_idf = normalize(tf_idf)
In [6]:
for i in range(5):
doc = tf_idf[i]
print(np.linalg.norm(doc.todense()))
In [58]:
%%time
from sklearn.cluster import KMeans
np.random.seed(5)
num_clusters = 25
# Use scikit-learn's k-means to simplify workflow
#kmeans_model = KMeans(n_clusters=num_clusters, n_init=5, max_iter=400, random_state=1, n_jobs=-1) # uncomment to use parallelism -- may break on your installation
kmeans_model = KMeans(n_clusters=num_clusters, n_init=5, max_iter=400, random_state=1, n_jobs=1)
kmeans_model.fit(tf_idf)
centroids, cluster_assignment = kmeans_model.cluster_centers_, kmeans_model.labels_
means = [centroid for centroid in centroids]
In [59]:
%%time
num_docs = tf_idf.shape[0]
weights = []
for i in range(num_clusters):
# Compute the number of data points assigned to cluster i:
num_assigned = np.sum(cluster_assignment == i) # YOUR CODE HERE
w = float(num_assigned) / num_docs
weights.append(w)
In [60]:
#cluster_assignment
covs = []
for i in range(num_clusters):
member_rows = tf_idf[cluster_assignment==i]
cov = (member_rows.multiply(member_rows) - 2*member_rows.dot(diag(means[i]))).sum(axis=0).A1 / member_rows.shape[0] \
+ means[i]**2
cov[cov < 1e-8] = 1e-8
covs.append(cov)
In [61]:
out = EM_for_high_dimension(tf_idf, means, covs, weights, cov_smoothing=1e-10)
In [62]:
out['loglik']
Out[62]:
In [23]:
# Fill in the blanks
def visualize_EM_clusters(tf_idf, means, covs, map_index_to_word):
print('')
print('==========================================================')
num_clusters = len(means)
for c in range(num_clusters):
print('Cluster {0:d}: Largest mean parameters in cluster '.format(c))
print('\n{0: <12}{1: <12}{2: <12}'.format('Word', 'Mean', 'Variance'))
# The k'th element of sorted_word_ids should be the index of the word
# that has the k'th-largest value in the cluster mean. Hint: Use np.argsort().
sorted_word_ids = np.argsort(-means[c]) # YOUR CODE HERE
for i in sorted_word_ids[:5]:
print('{0: <12}{1:<10.2e}{2:10.2e}'.format(map_index_to_word[i],
means[c][i],
covs[c][i]))
print('\n==========================================================')
In [24]:
'''By EM'''
visualize_EM_clusters(tf_idf, out['means'], out['covs'], map_index_to_word)
In [48]:
np.random.seed(5) # See the note below to see why we set seed=5.
num_clusters = len(means)
num_docs, num_words = tf_idf.shape
random_means = []
random_covs = []
random_weights = []
for k in range(num_clusters):
# Create a numpy array of length num_words with random normally distributed values.
# Use the standard univariate normal distribution (mean 0, variance 1).
# YOUR CODE HERE
mean = np.random.normal(0, 1, num_words)
# Create a numpy array of length num_words with random values uniformly distributed between 1 and 5.
# YOUR CODE HERE
cov = np.random.uniform(1, 5, num_words)
# Initially give each cluster equal weight.
# YOUR CODE HERE
weight = 1 / num_clusters
random_means.append(mean)
random_covs.append(cov)
random_weights.append(weight)
In [49]:
len(random_means), len(random_covs), len(random_weights), num_words, num_docs, num_clusters, random_means[0].shape
Out[49]:
In [50]:
out_random_init = EM_for_high_dimension(tf_idf, random_means, random_covs, random_weights, cov_smoothing=1e-5)
In [51]:
out_random_init['loglik']
Out[51]:
In [52]:
num_clusters
Out[52]:
In [63]:
out = EM_for_high_dimension(tf_idf, means, covs, weights, cov_smoothing=1e-5)
In [66]:
out['loglik']
Out[66]:
In [67]:
visualize_EM_clusters(tf_idf, out_random_init['means'], out_random_init['covs'], map_index_to_word)
In [ ]: